Jonathan Kadmon

Wednesday 4th of July 2018

Time:4.00pm

Ground Floor Seminar Room

25 Howland Street, London, W1T 4JG

Dynamics and computations in hierarchical neuronal networks

Many neuronal circuits in the brain (e.g., different sensory pathways, the hippocampus, and the cerebellum) are organized in a hierarchical fashion. However, the computational benefits of the layered structure are still not well defined, in particular when one considers that given enough neurons, a single-layer architecture can act as a universal function approximator [Hornik, 1991].

In this talk, I will introduce a simple multilayered perceptron network, generally considered as a plausible biological model. The input layer consists of high dimensional manifolds, each representing fluctuations of a prototypical stimulus. The difficulty of the decoding depends on the number and size of the manifolds. Neurons in the network are trained locally to perform binary classification with random labeling; namely, they fire in response to a unique subset of the prototypical stimuli and remain silent for all others. Using dynamic mean-field approximation, which becomes exact for large networks, I will show that emergent correlations between different representations limit the decoding performance of shallow architectures. These correlations do not vanish as the width of the layer is increased. In contrast, the layer-by-layer dynamics of a hierarchical structure can gradually remove correlations, allowing errorless decoding of the activities in the penultimate layer. For a given number of available neurons, I will find the optimal depth that maximizes the capacity, defined by the size and number of manifolds that can be decoded. Importantly, the framework is learning-rule agnostic, and training can be implemented using a broad range of local plasticity schemes. In particular, for hard problems in which the input manifolds are barely separable, the optimal depth of the network and its capacity show universal power-law scaling that does not depend on the underlying plasticity rule and supports the generality of the results.